Papers with Matryoshka Representation Learning
LLMs Meet Isolation Kernel: Lightweight, Learning-free Binary Embeddings for Fast Retrieval (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) embeddings are typically high-dimensional, leading to substantial storage and retrieval overhead. |
| Approach: | They propose a learning-free method that transforms an LLM embedding into a binary embeddable using Isolation Kernel (IKE). |
| Outcome: | The proposed method performs 16.7 faster retrieval and 16 lower memory usage than the original LLM embeddings while maintaining comparable accuracy. |
MIPIC: Matryoshka Representation Learning via Self-Distilled Intra-Relational and Progressive Information Chaining (2026.findings-acl)
Copied to clipboard
Phung Gia Huy, Hai An Vu, Minh-Phuc Truong, Thang Duc Tran, Linh Ngo Van, Thanh Hong Nguyen, Trung Le
| Challenge: | Existing approaches to train dense representations require explicit coordination of how information is arranged across embedding dimensionality and model depth. |
| Approach: | They propose a framework that trains Matryoshka representations using self-distilled intra-relational alignment and Progressive information chaining. |
| Outcome: | The proposed framework produces coherent and compact Matryoshka representations with significant performance advantages under low-dimensional models. |
Enhancing Lexicon-Based Text Embeddings with Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Recent large language models (LLMs) have demonstrated exceptional performance on general-purpose text embedding tasks. |
| Approach: | They introduce the first lexicon-based embeddings that consolidates the vocabulary space through token embeddation clustering to handle the issue of token redundancy in LLM vocabularies. |
| Outcome: | The proposed model outperforms dense embeddings on the Massive Text Embedding Benchmark (MTEB) it also supports efficient dimension pruning without any specialized objectives like Matryoshka Representation Learning. |